Support abort background compaction jobs. by xingbowang · Pull Request #14227 · facebook/rocksdb

xingbowang · 2026-01-08T19:45:13Z

Summary:
This adds a new public API to allow applications to abort all running compactions and prevent new ones from starting. Unlike DisableManualCompaction() which only pauses manual compactions and waits for them to finish naturally, AbortAllCompactions() actively signals running compactions (both automatic and manual) to terminate early and waits for them to complete before returning.

The abort signal is checked periodically during compaction (every 100 keys), so ongoing compactions abort quickly. Any output files from aborted compactions are automatically cleaned up to prevent partial results from being installed.

This is useful for scenarios where applications need to quickly stop all compaction activity, such as during graceful shutdown or when performing maintenance operations.

Test Plan:

Unit tests in db_compaction_abort_test.cc cover various abort scenarios including: abort before/during compaction, abort with multiple subcompactions, nested abort/resume calls, abort with CompactFiles API, abort across multiple column families, and timing guarantees
Updated compaction_job_test.cc to include the new parameter

Summary: This adds a new public API to allow applications to abort all running compactions and prevent new ones from starting. Unlike DisableManualCompaction() which only pauses manual compactions and waits for them to finish naturally, AbortAllCompactions() actively signals running compactions (both automatic and manual) to terminate early and waits for them to complete before returning. The abort signal is checked periodically during compaction (every 100 keys), so ongoing compactions abort quickly. Any output files from aborted compactions are automatically cleaned up to prevent partial results from being installed. This is useful for scenarios where applications need to quickly stop all compaction activity, such as during graceful shutdown or when performing maintenance operations. This also adds a new public API to resume compactions after the call to abort. Limitation: compaction service is not support. Test Plan: - Unit tests in db_compaction_abort_test.cc cover various abort scenarios including: abort before/during compaction, abort with multiple subcompactions, nested abort/resume calls, abort with CompactFiles API, abort across multiple column families, and timing guarantees - Updated compaction_job_test.cc to include the new parameter - Stress test

meta-codesync · 2026-01-26T16:46:36Z

@xingbowang has imported this pull request. If you are a Meta employee, you can view this in D91480994.

anand1976

Overall LGTM

anand1976 · 2026-01-28T22:58:33Z

db/compaction/compaction_job.cc

+      }
+      for (const std::string& file_path :
+           sub_compact.Outputs(is_proximal_level)->GetOutputFilePaths()) {
+        Status s = env_->DeleteFile(file_path);


Wouldn't a compaction with non-ok status get automatically cleaned up? Why do we need to explicitly do the cleanup here?

The normal cleanup path (Cleanup() in compaction_outputs.h lines 247-253) only abandons in-progress builders. This does not delete already-finished output files that were successfully written to disk.

When compaction runs with multiple subcompactions in parallel:

Subcompaction A completes successfully → produces finished SST/blob files on disk

Subcompaction B gets aborted (or the overall compaction is paused)

The overall compaction status becomes CompactionAborted or ManualCompactionPaused.
At this point, Subcompaction A's output files are fully written and finished on disk
But the overall compaction is aborted, so these files will never be installed to the LSM tree
Without explicit cleanup, these files become orphans on disk

Interesting. Thanks for the clarification. I think you're right. Is this a problem with subcompactions in general then? For example, if subcompaction B fails due to IO error, then there's no cleanup of subcompaction A's files. Not saying that needs to be addressed in this PR since its a separate issue, but is it something to be tracked?

I did some more investigation around this. There is another function FindObsoleteFiles that scan directories to find files not in any live version, and perform clean up, on compaction or flush failure. We could rely on that for rare failure such as IO error. For abort operation, we could switch to that as well. However, it would break resumable compaction, as FindObsoleteFiles does not know a compaction is resumable or not.

db/compaction/compaction_service_job.cc

anand1976 · 2026-01-29T00:39:11Z

db/compaction/compaction_job.cc

+    const uint64_t num_records = c_iter->iter_stats().num_input_records;
+
+    // Periodic cron operations: stats update, abort check, and sync points
+    if (num_records % kCronEvery == kCronEvery - 1) {


Nit: Can we avoid the % (or make kCronEvery a power of 2)?

anand1976 · 2026-01-29T00:54:52Z

db/db_compaction_abort_test.cc

+// max_subcompactions values
+class DBCompactionAbortSubcompactionTest
+    : public DBCompactionAbortTest,
+      public ::testing::WithParamInterface<int> {};


I would add a comment specifying what exactly the param is for

anand1976 · 2026-01-29T00:57:35Z

db/db_compaction_abort_test.cc

+  ConfigureOptionsForStyle(options, style);
+  Reopen(options);
+
+  // Use larger value size for Universal compaction to ensure compaction work


Could you elaborate a bit more? Why wouldn't it work? If not having a specific amount of work breaks timing of the test, it may not be ideal

I forgot to clean this up. We no long this special configuration after tuning the parameter. Remove the specialization.

anand1976

LGTM

anand1976 · 2026-01-29T18:11:06Z

db/compaction/compaction_job.cc

+      }
+      for (const std::string& file_path :
+           sub_compact.Outputs(is_proximal_level)->GetOutputFilePaths()) {
+        Status s = env_->DeleteFile(file_path);


Interesting. Thanks for the clarification. I think you're right. Is this a problem with subcompactions in general then? For example, if subcompaction B fails due to IO error, then there's no cleanup of subcompaction A's files. Not saying that needs to be addressed in this PR since its a separate issue, but is it something to be tracked?

db/compaction/compaction_service_job.cc

meta-codesync · 2026-01-29T18:54:05Z

@xingbowang has imported this pull request. If you are a Meta employee, you can view this in D91480994.

…1_23

meta-codesync · 2026-01-30T12:34:18Z

@xingbowang has imported this pull request. If you are a Meta employee, you can view this in D91480994.

meta-codesync · 2026-01-30T13:55:29Z

@xingbowang merged this pull request in 656b734.

Summary: This adds a new public API to allow applications to abort all running compactions and prevent new ones from starting. Unlike DisableManualCompaction() which only pauses manual compactions and waits for them to finish naturally, AbortAllCompactions() actively signals running compactions (both automatic and manual) to terminate early and waits for them to complete before returning. The abort signal is checked periodically during compaction (every 100 keys), so ongoing compactions abort quickly. Any output files from aborted compactions are automatically cleaned up to prevent partial results from being installed. This is useful for scenarios where applications need to quickly stop all compaction activity, such as during graceful shutdown or when performing maintenance operations. Pull Request resolved: facebook#14227 Test Plan: - Unit tests in db_compaction_abort_test.cc cover various abort scenarios including: abort before/during compaction, abort with multiple subcompactions, nested abort/resume calls, abort with CompactFiles API, abort across multiple column families, and timing guarantees - Updated compaction_job_test.cc to include the new parameter Reviewed By: anand1976 Differential Revision: D91480994 Pulled By: xingbowang fbshipit-source-id: 36837971d8a540cd34d3ec28a78bc94b582625b0

meta-cla bot added the CLA Signed label Jan 8, 2026

xingbowang force-pushed the comp_abort branch 2 times, most recently from b112174 to 405863c Compare January 23, 2026 18:16

xingbowang force-pushed the comp_abort branch from 405863c to 92ef881 Compare January 26, 2026 14:38

xingbowang marked this pull request as ready for review January 26, 2026 21:31

xingbowang requested a review from anand1976 January 27, 2026 05:02

anand1976 reviewed Jan 29, 2026

View reviewed changes

xingbowang added 6 commits January 29, 2026 06:08

address feedback

6760a34

support abort_and_resume_compactions_one_in

449f65f

fix build

9550630

allow abort error in compaction

e6c0a80

fix crash test failure

e636668

fix format

563df0f

anand1976 approved these changes Jan 29, 2026

View reviewed changes

Merge remote-tracking branch 'upstream/main' into comp_abort_rebase_0…

d85555d

…1_23

meta-codesync bot closed this in 656b734 Jan 30, 2026

facebook-github-bot added the Merged label Jan 30, 2026

Conversation

xingbowang commented Jan 8, 2026

Uh oh!

meta-codesync bot commented Jan 26, 2026

Uh oh!

anand1976 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

anand1976 left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

meta-codesync bot commented Jan 29, 2026

Uh oh!

meta-codesync bot commented Jan 30, 2026

Uh oh!

meta-codesync bot commented Jan 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants